The mapping of a television image on a graphics screen generally requires the image to be sampled and stored into a frame buffer. This allows a time base correction to be provided and, if necessary, a time compression of the television image in order to show both television and graphics images on the same screen. If the image should be positioned in any arbitrary sized window on a graphics screen, it should be either scaled up or down. The image itself may not be a full screen image, but just an arbitrary window inside the full view image. This requires a translation and scaling operation on the source image. Translation is a simple matter, and shall not be discussed in this paper. FIG. 1 shows how a TV image window, Is, is transposed to a graphics screen window, Id.
Scaling up a television image on a graphics screen poses a much harder problem than scaling down. When scaling down a source image to a smaller destination image, it is possible to simply ignore pixels in the horizontal direction and ignore scan lines in the vertical direction to achieve the correct sizing at the time of sampling. For example, to achieve a quarter sized window, every other pixel can be thrown away horizontally, and every other scan line vertically. This can be done simply at the time of sampling the source image into a frame buffer. However, for image expansion, it is necessary to either replicate pixels in a horizontal direction or replicate scan lines in a vertical direction to achieve the desired destination window size. This poses a problem as the bandwidth of the frame buffer needs to be increased. For example, if it were necessary to double the source image in both horizontal and vertical directions, the bandwidth of the frame buffer required to achieve the expansion needs to be quadrupled. The result of this is that either a faster frame buffer memory is necessary (notice that the speed of the frame buffer depends on the enlarging ratio), a FIFO deep enough to store away incoming pixels such that they are not lost, or a mixture of both.
It should be noted that as the magnification ratio increases, the image quality decreases. For example, when the magnification ratio is greater than two, the sharpness of the image is greatly reduced.
Nevertheless, for current mixed video/graphics display systems, the significance of magnifying television images are of considerable importance. It is especially important due to overscan problems which will be discussed below.
In conventional commercial TV systems, the active portion of the raster (i.e., between blank signals) overscans the viewing area of the CRT to prevent a black border under worst-case conditions of variations in yoke sensitivity, anode voltage, etc. (See "Television Engineering Handbook", McGraw Hill Company, 1986, p.13.177). The overscan requirement for consumer TV receivers goes somewhat higher than 10% of the full active area (See FIG. 2). This means that less than 90% of the active video image is usually shown on a TV screen. It is safe to say that video is overscanned not more than 15%. This number is taken into consideration when video is edited.
On the other hand, high-resolution graphics monitors do not use an overscan approach. Rather, they are underscanned in order to present all graphics image pixels on the screen. In other words, it means that a black border always surrounds the image.
The majority of applications in the multimedia area overlay a television image with graphics. The most widely used approach is to match 85% (in both linear directions) of the active television video with full lengths of the active graphics video line. With this approach a multimedia editor can be sure that whatever television movies or other materials are combined with graphics, the television image coordinates will correspond to the graphics image coordinates with reasonable accuracy, and no unnecessary information, which might be overlooked during editing, will appear on the screen.
The overscan requirement is also important for providing compatibility with previously developed multimedia programs. For example, millions of dollars are spent on such video processing programs such as IBM Infowindow, educational and presentation programs, where the overscan is taken into consideration. For a detailed description of the Infowindow product, reference should be made to one of the following publications describing same.
1) "Infowindow Guide to Operations" Order No. SK2T0297 and, PA1 2) "Infowindow Enhanced Graphics Adapter: Hardware Maintenance and Service Manual" Order No. SK2T0298, both are available from IBM Corp. Mechanicsburg, Pa. Any multimedia display adapter which does not address the overscan problem can not be used with Infowindow or Infowindow-like programs. Moreover, such an adapter can not be used with the television material edited first on standard television editing equipment.
In the case of the IBM Infowindow product, a special enhanced graphics adapter (EGA) monitor is used which provides for the overscan of video. It is, however, not a common graphics monitor. The approach of the present invention allows the use of a standard graphics monitor, providing the television image overscan by a special sampling approach. Such a monitor architecture is disclosed and described in the publication "IBM Infowindow Color Display" No. ZR23-6820 available from the IBM Corp. Mechanicsburg, Pa.
One possible solution to achieving overscan is to choose a television image sampling frequency higher than that of the graphics video clock frequency. E.g., if IBM PS/2 VGA has a video clock frequency of 25 Mhz which corresponds to 640 pixels on the active portion of horizontal scan line, the video sampling frequency should provide 640 pixels on the active portion of the underscanned television horizontal scan line. Therefore, a total of 752 samples are required per scan line to achieve 640 pixels of underscanned samples (e.g., 7524859). Hence, 640 pixels of the sampled video image will correspond to 640 graphics pixels exactly, and the overscan requirement is satisfied.
This approach, however, does not work well when using the standard digital television sampling frequency. Frequently, the television image is decoded and sampled using standard digital television techniques. It provides a cheaper solution, better image quality, and easier control over the brightness, sharpness, hue, etc. Unfortunately, the CCIR 601-1 recommendation for digital television encoding and transmission (See "Handbook of Recommended Standards and Procedures, International Teleproduction Society", 1987, p. 62), which is widely used in the television industry, prescribes a sampling frequency of 13.5 Mhz. It gives a total of only 720 samples on a television scan line. With 15% overscan, it allows only 612 pixels as shown on FIG. 2. The present invention comprises a system for mapping 612 samples of the television image (pixels) onto a larger number of graphics pixels.
Clearly, the solution which preserves the sampling rate is to increase the number of samples after sampling has been done. Using the standard sampling rate of 13.5 Mhz, if the number of graphics pixels is 640, then the expansion ratio should be 640/604. However, this number is not a power of two, and the expansion can not be done in a simple way, like replicating every pixel. Another consideration is that if the graphics adapter has several modes with a different number of pixels in the horizontal line, e.g., 320, 640, 720 pixels in the case of a VGA graphics adapter, (See, for example, IBM PS/2 Model 80 Technical Reference #68X2256 available from the IBM Corp. Mechanicsburg, Pa.) the scaling ratio should be programmable. In a window environment, the expansion ratio should ideally be selectable to be any rational number defined by the size of the window.
This situation is even more complicated by the specific coding scheme of digital television. The standard television coding schemes, either NTSC, PAL, or SECAM are all based on luminance/chrominance (Y/C) representations, rather than RGB, which reduces the bandwidth of the composite video signal and memory required to store the image frame. Furthermore, some digital television chips already in production use a time multiplexing technique to reduce the bandwidth required for chrominance information. For example, Philips provides digital television chips (See, for example, "Digital Video Signal Processing" Philips Components Manual No. 9398 063 30011) in which luminance bit rate versus color bit rate is 4:1. Compared to 8 bits of luminance information per sampling clock, only 4 bits of chrominance are generated (2 bits for B-Y and 8 bits of R-Y) as shown in FIG. 3. Thus it takes four clock cycles to transmit a complete chrominance values (8 bits of B-Y and 8 bits of R-Y). This further complicates expansion in the horizontal direction since due to the time multiplexing, it is not possible to simply replicate pixels. Notice that for chrominance, the smallest horizontal resolution is 4 pixels wide. It is necessary to keep the synchronization of the chrominance bits over a period of four system clock cycles, and failing to do so will result in corrupt color on the destination screen for all pixels that are out of synchronization.